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Abstract 

Many satisfiability modulo theories solvers implement a variant of the DPLL(7“) framework which 
separates theory-specific reasoning from reasoning on the propositional abstraction of the formula. Such 
solvers conclude that a formula is unsatisfiable once they have learned enough theory conflicts to derive 
a propositional contradiction. However some problems, such as the diamonds problem, require learning 
exponentially many conflicts. We give a general criterion for establishing lower bounds on the number of 
theory conflicts in any DPLL(7“) proof for a given problem. We apply our criterion to two different state- 
of-the-art symbolic partial-order encodings of a simple, yet representative concurrency problem. Even 
though one of the encodings is asymptotically smaller than the other, we establish the same exponential 
lower bound proof complexity for both. Our experiments confirm this theoretical lower bound across 
multiple solvers and theory combinations. 


1 Introduction 

Many high-level verification tools rely on satisfiability modulo theories (SMT) solvers to dis¬ 
charge verification conditions in a variety of first-order logic theory theories. State-of-the-art 
SMT solvers decide such problems by implementing variations on the DPLL(7') framework. 
The DPLL(7') framework integrates a theory-specific solver with efficient search over the 
propositional abstraction of the formula. For this, DPLL(7~) uses a propositional (SAT) solver 
that searches for a satisfying assignment to the propositional abstraction of the formula. When 
such an assignment is found, a theory solver checks that this propositional assignment is 
theory consistent. If it is not, a theory conflict (or T'-conflict) clause is added, summarizing 
the inconsistency and preventing the SAT solver from exploring this part of the search space 
again. The process continues until either a theory consistent satisfying assignment is found, 
or a contradiction can be derived purely on the propositional level using the learned theory 
conflicts. While usually efficient in practice, there are well-known problems, such as the "dia¬ 
monds problem" |21J, on which the DPLL(7') framework cannot derive a contradiction using a 
pol 5 momial number of theory conflicts. This issue has resurfaced in recent work on worst-case 
execution time (TSl. This limitation stems from the fixed alphabet of the DPLL(7') theory con¬ 
flicts. Despite work on addressing this inherent inefficiency, it is still an open problem 171 1221 . 

In this paper, we prove a general theorem for establishing lower bounds on the number of 
T'-conflicts in the DPLL(7~) calculus 1191 required to prove that a given formula is unsatisfiable. 
The theorem relies on the notion of non-interfering critical assignments: propositionally satisfying 
assignments that contain disjoint 7~-conflicts. To the best of our knowledge, this is the first 
attempt at establishing a general framework for establishing lower bounds for DPLL(7') proofs. 
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We apply this theorem to study the DPLL(7') proof complexity of proving a safety property 
of a simple, yet challenging concurrency problem. The problem appears in the software 
verification competition (SV-COMP) and is of broad historical interest l20l[TT]| . We focus on 
encodings recently implemented in a bounded model checker |3| because they have been 
successfully used to find concurrency-related bugs in software such as the Apache HTTP 
server, PostgreSQL and the Linux kernel [3]. Informally, these encodings symbolically model 
a certain partial-ordering between memory accesses, similar to the happens-before relations in 
distributed systems Ill6l . 

Contributions. The main contributions of this paper are as follows: (1) we give a new result 
for establishing lower bounds on the size of DPLL(7') proofs of unsatisfiability; (2) we propose 
a new problem challenge for the SMT community, whose solution is directly relevant to finding 
concurrency-related bugs in software; (3) we establish a factorial lower bound on the size of 
DPLL(7') proofs of unsatisfiability for this challenge problem; finally, (4) we experimentally 
confirm the hardness of this problem. 

Organization. We prove the lower bound theorem in [section 21 We introduce the problem 
challenge and explain how to generate two equisatisfiable partial-order encodings in isection 31 
Given these encodings, we formalize the DPLL(7') proof size complexity of the challenge 
problem Jsection 4ll and experimentally confirm its complexity dsection 5ll . We conclude with 
a discussion of related work and future research directions in isection 61 


2 Non-interfering Critical Assignments 

In this section, we give a general theorem for establishing lower bounds on the number of 
T'-conflicts in all proofs that a formula (p is unsatisfiable in the DPLL(7') calculus 1191 . The 
theorem is based on the notion of sets of non-interfering critical assignments for cp. 

We assume readers are familiar with standard notions from SMT such as T'-conflicts, 7~- 
validity, 7~-lemmas, DPLL(7~), etc. In DPLL(7'), a proof of unsatisfiability for a T'-formula 
consists of a combination of learning T'-valid lemmas and performing resolution steps on the 
propositional abstraction, until the empty clause is derived. As in 1191 , we restrict the proofs 
to work over the fixed alphabet of 7~-atoms in the input formula and that all 7~-lemmas are 
clauses. We use a simplified view of the DPLL(7') calculus 1191 that only uses two rules: (i) 
propositional resolution (Res) and (ii) learning T'-valid clauses over the literals of Ji (T'-learn). 
We ignore T'-propagation and splitting-on-demand IQ. 

Notation. We fix a set of propositional variables A and use £ to denote literals over this set. 
A clause C is a set of literals interpreted as their disjimction. The empty clause □ denotes false. 
A partial assignment M is a set of literals that does not contain both a variable and its negation. 
Partial assignments are interpreted as a conjimction A^eM^ are always propositionally 
consistent. An assignment M is a partial assignment s.t. for all c e A either i; e M or -ii; e M. 
The negation of a clause is a set of literals -iC = I ^ e C} and is interpreted as a conjunction. 

The propositional abstraction function _® is an injective map from into A. The T'-literals, 
written are the set of literals over We lift _® to work over T'-literals and sets of T- 
literals. We denote by L a T'-valid clause over X.?i, Nr VteL N denote a T'-conflict. 

A 7~-conflict is a set of T'-literals whose conjunction is T'-unsatisfiable, -iL Nr n. A minimal 
T'-conflict has the additional property that every strict subset is T'-satisfiable. 
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Proofs. We assume the input 7~-formula (p has already been converted to CNF and is repre¬ 
sented as a finite set of clauses Ci,... over the variables in X, the set of T'-atoms and the 
boolean abstraction function _® : Ji ^ X. A Fixed-Alphabet-DPLL(7') proof has the form: 


Cl / • • • / C(j/ • • • / Cj ^,... / Cj^ izi 


where each Cjt for a < k < p is derived from a previous clause using either the resolution rule 
(Res) or theory learning (T'-learn). Let C, Cj denote propositional resolution on 

Ci,...,Ck LQ \=r V 

Cl. 


t€L 


t 


7~-learn 


Cl,..., Cjt 1 < 1 < / < A: { & Ci G Cj 
Cl,, Ck, Ci Cj 


Res 


The rule T'-learn adds a new clause L® that corresponds to the propositional abstraction of a 
T'-valid clause. Clauses derived by T'-learn are called 7~-lemmas. T'-learn is more general 
than Lazy Theory Learning fl9l . which requires the literals to be in the partial assignment. 

Critical Assignments. Given a T'-formula cp, an assignment M is critical if it satisfies the initial 
propositional abstraction of cp (i.e., M |= A^Li Ci) and there is exactly one minimal T'-conflict 
-iL such that -iL® c M. We denote by Q a set of critical assignments for (p, all of which can be 
enumerated as Mi, ..., M|q| and where -iLi denotes the minimal T'-conflict for M;. We say that 
Q is non-interfering whenever, for all M, Mj in Q, -iL® is not a subset of Mj. In other words, 
no two assignments in Q contain the same T'-conflict. 

Lemma 2.1. Let Mhe a critical assignment for cp with the minimal T -conflict -iL, and Tlbe a Fixed- 
Alphabet-DPLL(T) proof that cp is unsatisfiable. There is a T-learn application Cr e TI such that 
-L® c -nCk c M. 


Proof. The assignment M does not satisfy the last clause C^ = □ in IT. Therefore, there is some 
first clause Cjt that M does not satisfy in IT. The clause Ck cannot be an input clause as M C; 
for 1 < i < a. Additionally, Ck cannot be the result of Res: since Ck is the first rmsatisfied clause, 
all M 1= Ci for i < k, and resolving C; and Ck for i + i' <k results in a clause satisfied by M. Thus 
Ck must be the result of a 7 ~-learn application and Ck- Since M is an assignment which 
does not satisfy Ck, M must contain the negation of all literals in Ck- Equivalently, ->Ck £ M. 
Let T be the 7~-lemma corresponding to Ck'. Ck = T^. As -iL® is the unique minimal subset of 
M that maps to a minimal theory conflict, L c T. Therefore, -iL® c -iCk £ M. □ 

Intuitively Lemma l2T] states that, for each critical assignment M, the proof of unsatisfiability 
must contain a clause, derived by T'-learn, which rules out M as a model of cp in the theory 7~. 

Theorem 2.2. Let cp be an unsatisfiable T-formula, and let Q be a non-interfering set of critical 
assignments for cp. Then all Fixed-Alphabet-DPLL{T) proofs that cp is unsatisfiable contain at least |Q| 
applications ofT -learn. 

Proof. Let IT be any Fixed-Alphabet-DPLL(7') proof. We will show that there exists a surjective 
partial map from 7~-lemmas in FI onto critical assignments in Q that contain the same 7~- 
inconsistency. We examine the set of partial maps F over {a, f] indices such that F{k) = j only 
if L® £ Ck and Q is a T'-learn application. Let the partial frmction F* be a partial function that 
maps onto the maximal number of distinct M e Q among all such maps F. If F’ maps onto all 
elements in Q, there are at least |Q| applications T'-learn in IT. If |Q| = 0, the property trivially 
holds on n. 
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For the remainder of this proof, assume that |Q| > 1. Suppose for contradiction that F* is 
not surjective. We can then select some critical assignment My such that for all k e (a, jS] either 
k is not in the domain of F’ or ¥*{k) + j. 

By Lemma [2.11 there exists a T'-learn application Q e fl such that -iL® c -.Q c My. As 
L® £ Cjt, we know that it is possible for F* to map Q to some Mm e Q. As F* is maximal and 
there is no conflict mapped to My, ¥*{k) = m for some m + j. By the construction of F*, L® c Q. 
Recall that -iQ c My. Thus -iL® c -iQ c My. As My contains both -iL® and -iL® for some 
distinct Mm in Q, this contradicts the assumption that Q is non-interfering. 

We can now conclude by contradiction that F’ maps some clause that is the result of T'-learn 
in n onto each M e Q. Therefore n contains at least |Q| applications of 7~-learn. □ 

There are many instances in the literature of diamond benchmarks for which exponential 
lower bounds on the number of T'-conflicts have been given l2n [7l[171121 [131 . Theorem l2.2l can 
be seen as a generalizafion of the lower boimd arguments for the diamond benchmarks. The 
rest of this paper is devoted to a novel application of Theorem l2.21 


3 Challenge problem 

In fhis section we present a challenge problem based on the fpk2013 SV-COMP concurrency 
benchmark [1]. This problem was first introduced in 1976 to illustrate the need for auxiliary 
variables in compositional proof rules for concurrenf programs I2D] , and most recently it has 
resurfaced as a challenge problem for automated verification tools fTTl . Consider the following 
simple shared memory program wifh N + 1 threads and a shared memory location x: 


Thread Tg 

Thread Ti 

Thread 

local Vo : = [x] 

assert(z;o < N) 

local vi : - [x] 
[x] : = z^i -h 1 


local vn ■ - [^] 
[x] :^vn + 1 


The memory at location x is denoted by [x]. We assume that [x] is initially 0. Each thread T; 
reads the value at memory location x into a CPU-local register u,. For i > 1, thread T, overwrites 
the memory at location x with the new value Vi + 1. For the rest of the paper, we denote the read 
of memory location x in Tq by Tassert- The reads and writes on memory location x in thread T, for 
i > 1 are denoted by r; and a;,, respectively. We follow the SV-COMP convention and assume 
sequential consistency IfTSl . Therefore, if we just consider the concurrent program Ti || T 2 , we 
get the following six interleavings of shared memory accesses: (1) fi; wi) r 2 ', W 2 , (2) r\) 12 ) w\) W 2 , 
(3) ri; r2) VJ2) vj\, (4) r2) n; zvi; ZV2, (5) r 2 ; ri; W2; wi, (6) r2; ZV2; n; zvi. The differenf orders can result 
in different final values of [x]. For example, ri;zui; r 2 ; W 2 results in the final value 2 at memory 
location x, whereas ri; r 2 ', vjy, W 2 results in the final value [x] = 1. 

We want to check that the assertion cg < N in thread Tg cannot be violated. Intuitively, this 
assertion holds because each of fhe other N threads increments [x] at most once. For a fixed 
N, we wanf to prove this automatically using bounded model checking. While it is easy to 
automatically prove this property on each separate interleaving, the number of interleavings 
grows exponentially {{2N + 1)! -r 2^). Next, we explain how to generate symbolic partial-order 
encodings that formalize all interleavings as a single quantifier-free SMT query. 

Partial-order encodings. We formalize fwo quantifier-free and equisatisfiable partial-order 
encodings of a concurrency semantics called SC-relaxed consistency ITU : a cubic-sized en¬ 
coding (£^) and a quadratic-sized encoding (£^). The formula generated by each encoding is 
satisfiable if and only if fhe safety property in the shared memory program can be violated. 
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PPO = ^ {{guard(e) A guard(e')) (Cg < Ce') | e,e' e E: e «: e') 

WW[x] — ^ Oil)' V ^ ^w) ^ ^ Sjtj' I W, W C X A IP ^ IP } 

RW[x] = y\^ {Cz(, <; Cr V c,- <; Ca; I IP e W;f A r e R^j 
RFtoI^;] - f\ {guard{r) ^ V ISio = s,- 1 ip e W^;} | r e R;c) 

RF^[x] = ^ {(Sa; = Sr) => (guard{w) A val(w) = rVy A Cu, < Or) \ r e A w e 

FR[x] = {(Sa; — Sr A Ow ^ Ca;^ A gUClvdiw )) (Cr ^ Ca?') [ IP/ IP C A T G R^} 

= /\ {rFto[^] a RF^[x] a FR[x] a WW[x] a RW[x] | x e (ADDRESS)] A PPO 

RF^[x] = ^ {(Sa, = Sr) => (Ca, = sup^ A guard(iv) A val(w) = rVr A c,a -< Cr) I r e R;,- A IP e Wj.) 

SUP[x] = ^ {(Czo < Cr A guard{w)) =» (Ca; < sup^) I r e Rj: A IP e W,:) 

6^ = [\ {RFto[^] a RF^[x] a SUP[x] a WW[x] A RW[x] | x e (ADDRESS)] A PPO 

Figure 1: Given a shared memory program structure P = (E,<i^,val, guard), £? and encode 
P's SC-relaxed consistency [14| with a cubic and quadratic number of constraints, respectively. 


To get 8? and &, we make four simplifying assumpfions about the program P imder 
scrutiny: (i) P's weak memory concurrency semantics equates to SC-relaxed consistency [14j; (ii) 
P is well-structured; (iii) all loops in P have been unrolled so that the only remaining control-flow 
sfatements in P are if-then-else branches; finally, (iv) every shared memory location accessed 
by P is known at compile-time. Avoiding these restrictions is beyond the scope of fhis paper 
thaf concerns ifself wifh SMT solvers rather than program analysis techniques. 

The formulas generated by both encodings 8? and 8? have three parts: (i) clock constraints 
that partially order memory accesses, similar to the happens-before relation in distributed 
systems 1161 : (ii) value constraints that determine what values are read or written by the program 
if fhose clock constraints hold; and (iii) selection constraints that associate each read to a specific 
write event. Our symbolic partial-order encoding is therefore parameterized by three theories: 
7~c for encoding fhe clock constraints, TV for encoding constraints on the symbolic program 
values, and Ts for encoding selection consfrainfs. We assume fhat TV's signafure includes 
sfrict and non-sfrict partial-order relations, denoted by < and <, respectively. We also assume 
that TVV signature can encode a decidable fragment of common machine arifhmefic such as 
bitvecfor or Presburger arithmetic. 7s is an uninterpreted theory. 

Definition 3.1. A shared memory program structure is a tuple P = (E, val, guard) where E is a 
finite set of events, is a partial order on E, val: E T'y-terms and guard : E —> 7'v-formulas. Let 
(ADDRESS) be the set of memory locations. We assume that the set of events E in P can be partitioned 
into reads Rx and writes Wx on memory location x e (ADDRESS). Given an event e in E, let Cg and 
Sg be a T'c-'oariable (clock variables) and Ts-variable (selection variables), respectively. For each 
read r e R, let rVg be a unique Ty-variable, called read variable. The function val maps a write event 
w eW to a Ty-term val(w) built from read variables. 

The partial order ^ is the preserved program order (PPO) ||4l|3. The intuition behind PPO 
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is that it determines which events cannot be reordered in any execution of the program. For 
sequentially consistent programs, the preserved program order corresponds to the order of 
insfructions in each thread. Note that (E, can be relaxed for weaker forms of consistency 
such as TSO, e.g. |3|. Intuitively, given an event e in E, guard{e) denotes the necessary condition 
for e to be enabled. The equality Sa; = Sr in the theory Ts means that a read event r is 'selected' 
so that its input value is equal to the output of a write event w. That is to say, when Sa; = Sr 
holds, the TV-variable rv,- is equal to the term val{zu). 

Example 3.2. The program described in \sectiori3\ for N = 2 corresponds to the following: 

• E = {winit,ri,Wi,r 2 ,W 2 ,rassert] IS partitioned into R^c = Wrt) and = {wi„it,Wi,W 2 ] 

where x e (ADDRESS) is the concrete memory location accessed by threads Tq, Ti and T 2 . 

• According to PPO: Wi„it ^ ri ^ wi, Wmu «c r 2 ^ W 2 , and Wi„it rassert- 

• The val function is defined as val{winit) - 0, val{wi) = ry^ + 1 and val{w 2 ) - rVrj - 1 -1. 

• Since the program has no if-then-else statements, guard{e) = true/or all events e in E. 

Figure [T] shows how to generate the cubic-size and quadratic-size partial-order en¬ 
coding for a given shared memory program structure P = val,guard). The first four 

formulas, PPO, WW[x], RW[x], and RFto[^]/ are shared by and £^. The constraint PPO 
encodes the preserved program order The remaining constraints are with respect to some 
concrete memory location x. To model the information flow in the program, we encode a form 
of the read-from relation ||4ll3. For a fixed memory location x this relation defines a function 
from R;c to W^. We model this through the selection variables s^ and Sh,, for each read r e R,: 
and write w e together with the equality Sr = Sw The intuition is that the value of a write 
event w e VJx is observed by a read event r e R^ iff = Sw The RFto constraints ensures that 
at least one such equality holds for every read. WW encodes that all writes on the same shared 
memory location are totally ordered in the happens-before relation and cannot have the same 
selection value, and RW encodes that every read r and write w on the same shared memory 
location satisfy that r happens-before w, or vice versa. Note that if -< is a tofal order, then WW 
is equivalent to the clock and selection variables being distinct. (In practice, the Sa, variables 
are optimized out as distinct constants.) The same is not true for RW because two reads can 
have the same clock variables. 

The main difference between £^ and £^ is how they encode values being overwritten in 
memory. A read r in R^ can read from a write w in W;^ if w is the most recent write to x that 
happens before r. In the case of £^, this is encoded by FR which corresponds to the 'from-read' 
axiom sia , also known as the 'conflict relation' |8j . This formula infroduces a cubic number of 
constraints. By contrast, £^ encodes the SUP constraint that requires only a quadratic number 
of constraints. For this, SUP introduces a new variable sup^ for every read r in R;^ to encode 
the least upper bound (supremum) of all writes in Wj, that happen-before r. Since the set 
{Cio 1 w e Wj:|, for all memory locations x, is totally ordered with respect to < in 7c by WW[x], 
sup,, is the maximum of all wrifes in Wj, fhat happen-before r in Rj, according to <. It was 
previously shown in IIT4l Theorem 4] that for a given shared memory program strucfure P the 
formulas £^ and £^ are equisafisfiable. 


4 Lower Bounds for Quadratic and Cubic Encodings 

We show fhat the challenge problem from section |3] requires DPLL(7~) to enumerate at least 
N! theory conflicts before it finds a proof of unsafisfiabilify, for eifher of fhe £^ or £^ encoding 
where N is fhe number of threads. 


6 




A Concurrency Problem with Exponential DPLL(T) Proofs 


Hadarean, Horn, and King 


^ ^ ^rassert ^ ^ ^ ^ yA^ Cjy ^ Cu)' A ^ A C^jj ^ Cj- A 

zv,zu'e\N,zu^w' zveV\l,reR 


2=1...N 


PPO 


PPO 


WW[x] 


RW[x] 


y/^ (Szy — Sr) ^ Cjy ^ C^- A — S^-) ^ 0 — TV^- A ^A\^ (Szy^ — S^-) TV^. + 1 — 


rv. 


z27€W,r€R 


rcR 


2=l...N,rGR 


RF^[x] 


RF^[x] 


yAy (Szy — S^- A Cztf ^ Czt?') ^ C^- ^ Cztf'’ A ^Ay \y^ 


z(;,iy'GW,r€R 


Szz; — Sr 


r€R \zv€\N ) 


RF^M 


A rVr ^ > N 

' assert 


assert{i?o^^^) 


FR[a:] RFxo[J^] 

Figure 2: The encoding for the challenge problem (when -< is total). 


We begin by constructing a formula that encodes the challenge program using the 
encoding. As £^ is not directly in CNF, we perform the following simplifications in order to 
apply Theorem 12.21 (i) all of the guards guard{e) are ignored because they always evaluate 
to true, and (ii) implications are distributed across conjunctions in the RF^[a:] constraints 
[A (B A C) iff (A => B) A (A ^ C)]. We also assume that ^ is a total order in 71;, and that 
TV is either bit-vector, Presburger, or real arithmetic. We denote by T~ the standard combined 
theory Tq + TV + T^. Figure |2] shows the resulting quantifier-free T'-formula, denoted by . 
Note that cp^ is in CNF if we interpret implications in the obvious way. Note that in the RF^[x] 
constraints, each val{w) term has been replaced by either 0 or rv^ -f-1. 

Let Sn be the set of all permutations over [1, N]. Consider the following sequence of events 
that can be constructed from the permutation function ti in Siv: 


(7(71) : Winitf N(l)/ ^Ti(l)/ ^7z{2)/ ^ 7 t( 2 )/ • • • / ^n{N)r ^7z{N)/ ^assert' 

The run of a{n) corresponds to satisfying the following clock and selection constraints: 




Szp-,x — s 


r„m' /\ 




~ and - S, 


IPn(N) 


2=1...N-1 


with distinct values for all Sw variables. A first-order variable assignment Vn can be constructed 
to satisfy the above constraints. (An explicit construction of v-n and proofs for Lemma [4.11 and 
Theorem 14.31 are given in Appendix |3) For each TV or TV literal (, we include in an 
assignment Mn if t holds under v„. Consider the following TV-conflict: 

-^L„ = = 0 } u + 1 = V = 1 .. .N - l} u {rv,„,^ + 1 = rv,_,} u {rv,_, > N). 

Note that each f e -iL„ is unit-propagated by the TV and TV literals already in M„ on the 
propositional abstraction of (p^. We add -iL® to Mn- The remaining TV equality atoms in (p^ 
are added negatively. Now Mn satisfies the propositional abstraction of cp^. 

Lemma 4.1. The assignment Mn is a critical assignment for (p^ with the theory conflict -^Ln- 

Theorem 4.2. All Fixed-Alphabet-DPLL(T) proofs for (p^ contain at least N\ applications ofT -learn. 
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Proof. Let Q = [Mn | ti e S;v j. For each pair of distinct n and n' in S]v, there is some adjacent pair 
of events with a different order in a{n) and cr(7i'). Select k so that ^n(k+Y^ + ifn'(k), 

The literal +1 = is in ->Ln and is not in Mn'- Thus -iL® is not a subset of Mn', 

and Q is non-interfering. The lemma follows directly from Theorem 12.21 □ 

Theorem 4.3. Let (p^ be the & encoding of the challenge problem. All Fixed-Alphabet-DPLLCT) proofs 
that (p^ are unsatisfiable contain at least N\ application ofT -learn. 

An important difference between the diamond benchmarks and this problem is that for 
diamonds it is reasonable to describe all minimal T'-conflicts as they each also correspond to 
critical models. For the fkp problem, the encoding is more complex, and there are other classes 
of T'-conflicts. The set Q identifies those 7~-lemmas that must appear during solving. 


5 Experiments 

In this section, we give experimental results that confirm the lower boimds on the DPLL(7') 
proofs for the two encodings of the problem challenge d section 3l l. Our experiments are carried 
out along three dimensions: we use four SMT solvers (Boolector v2.0.6 Q, CVC4 2015-03-14 |[5l, 
Yices v2.3.0 m, and Z3 2015-03-29 flSl ), and we evaluate both the cubic-size and quadratic-size 
encoding (£^ and £^) with respect to four different SMT-LIB theory combinations. 

We performed all experiments on a 64-bit machine running GNU/Linux 3.16 with 2 Intel 
Xeon 2.5 GHz cores and 4 GB of memory. The timeout for each individual benchmark is 1 hour. 
Recall that £^ and £^ are parameterized by three theories, 7c, T's and TV. We experiment with 
the theory of reals TV, the theory of integers TV, and the theory of bit-vectors Tbv In our 
experiments, we instantiate (7c,T's,TV) to four configurations such that 7c - T's'. 

(1) "real-clocks-int-val": (TV, TV, TV), (3) "bv-clocks-int-val": (7Vv,TVv,TV), and 

(2) "real-clocks-bv-val": (7V,7V,7Vv), (4) "bv-clocks-bv-val": (7Vv,TVv,TVv)- 

CVC4 and Z3 were run on all benchmarks. Boolector is only used on the fourth configuration, 
i.e. purely TVv benchmarks. Yices was run on the "real-clocks-int-val" and "bv-clocks-bv-val" 
configurations. We further distinguish between the SMT-LIB benchmarks by labelling them 
with £^ or £^. For example, 'real-clocks-bv-val-£^' identifies benchmarks generated with the 
cubic encoding in which 7c, T's and TV are respectively instantiated as TV, TV, and Tbv 

For all the "’^-bv-val" benchmarks (except CVC4 for "real-clocks-bv-val"), the solvers are 
essentially encoding the problem in propositional logic and using a SAT solverlJ The process of 
encoding into propositional logic {bit-blasting) enables the solver to learn clauses not necessarily 
expressible in the original alphabet of the input atoms. We therefore call these solver and 
configuration pairs bit-blasted combinations. All other solver and configuration pairs are called 
DPLL(T~) combinations. The DPLL(7') combinations are the "*-int-val" configurations, and the 
nm of CVC4 on "real-clocks-bv-val" 0 DPLL(7') combinations use Fixed-Alphabet-DPLL(7') 
proofs, whereas bit-blasted combinations generally do not. 

Given an instantiation of (Tc,Ts,TV), we separately encode the £kp2813-unsat concur¬ 
rency benchmarks with £^ and £^ for all N e [3,9]. There are a total of 56 different unsatisfiable 
SMT-LIB benchmarks. The size of each benchmark depends on N and whether we used 
£^ or £^. For example, for N = 9, the total number of symbolic expressions in £^ is 4085, 
whereas £^ yields only 1604 symbolic expressions. 
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^ CVC4 was run with the flag --bitblast=eager on "bv-clocks-bv-val" benchmarks 1121 . 
^In this configuration CVC4 does not eagerly reduce Tbv to SAT. 
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Number of threads (N) 







cvc4-real-clocks-int-val-{£^, 
cvc4-real-clocks-bv-val-{fi^, £^) 
cvc4-bv-clockS“int-val-{£^, 
cvc4-bv-clocks-bv-val-{£^, 
z3-real-clocks-int“Val-{£^, &] 
z3-real-clocks-bv-val-{fi^, £^| 
z3-bv-clocks-int-val-{fi^, fi^| 
z3-bv-clocks-bv-val-{£^, £^1 
yices-real-clocks-int-val" {£^, fi^) 
yices-bv-clocks-bv-val-{£^, fi^| 
boolector-bv-clocks-bv-val-{£^, £^) 


Figure 3: Experimental results for the fkp2013-unsat benchmark using four SMT solvers and 
four SMT-LIB theory combinations. The graph shows the factorial growth of the number of 
SAT conflicts in both the cubic-size and quadratic-size partial-order encoding as N increases. 


Figure |3] charts the number of conflicts reported by each solver during execution^ Execu¬ 
tions that exceeded the time limit of 1 hour are not included. The x-axis corresponds to N. The 
y-axis corresponds to the number of conflicts generated by the solver and has a logarithmic 
scale. The legend for the chart groups together both the (bold lines) and (thin lines) for a 
solver and theory specification. These are further grouped into bit-blasted benchmarks (dotted 
lines) and DPLL(7') (solid lines). We also plot N! as a black line. The goal of the Figure|3]is to 
convey the overall trends instead of compare individual data points. 

We examine the number of SAT conflicts as it is a uniform and readily available statistic that 
is a lower bound on the number of proof steps taken by each solver. Across all combinations, 
the number of conflicts observed is above the N\ line. Thus the N\ theory conflict lower 
bound proofs given in section |4] holds for the DPLL(7') combinations. Our theoretical lower 
bounds do not extend to the bit-blasted combinations. Nevertheless, our experiments show 
that the number of SAT conflicts are two orders of magnitude higher than N! for bit-blasted 
combinations. We therefore conjecture that a similar N! lower bound exists for Res proofs 
for the bit-blasted combinations. We also examined CVC4's more detailed statistics on the 
DPLL(T') combinations. We confirmed that the number of 7V-conflicts is always above N! on 
the DPLL)?”) combinations. 


6 Conclusion 

In this paper, we have demonstrated a theoretical factorial lower bound on the number of 
T'-learn applications in all DPLL(7') proofs for a challenge problem of historical interest using 
two state-of-the-art encodings. Our encodings are most closely related to ll3l [T4l . Experiments 


® Elapsed time and memory usage for the experiment is available in AppendixlBl 
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confirm the theoretical lower bound for DPLL(7~) proofs and show a strong relationship to the 
number of SAT conflicts in REs-proofs for bitblasted bitvector encodings. Both the theoretical 
relationships and the empirical relationships hold over a cubic and a quadratic & encoding. 
Our experiments are therefore particularly significant for state-of-the-art tools such as CBMC 
(which implements a variant of £^). We believe that the kind of analysis we have undertaken 
throughout this paper provides an important diagnostic practice in the development of SMT 
encodings. Future work will focus on handling the value constraints for partial-order encod¬ 
ings of weak memory concurrency and improving the performance of the SMT solvers on such 
benchmarks by moving outside of Fixed-Alphabet-DPLL(7') proofs. 
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A Proofs for Lower Bounds 

This section gives a more formal derivation for and the concepts discussed in section 31 
and proofs for Lemma l4d] and Theorem l4.3l We use p, v |= to denote that a E-structure ji and 
a variable assignment over p satisfies a E-formula (p. 

Let /i be any (7c + 7s + 7V)-structure with the additional constraint that Tc, 7s, and TV 
sorts are mapped to domains with cardinalities at least |E|, N -f-1, and N -f-1 respectively. Such 
a structure exists rmless TV is bit-vectors and the bit-width is insufficiently large. We now 
construct a first-order variable assignment Vn over TV and TV variables to match a{n). Let 
xi <1* . •. x\E\ be any arbitrary chain in the 7c domain of p, and let {yo,y\,.,yn) be an 
arbitrary enumeration of A1 -t-1 distinct elements in the 7s domain. Both the x; chain and the 
y, sequence exist as the cardinalities are large enough. We now assign the Ce and Sg variables. 


Xi 






X2i 

^ ~ ^n{i) 

VniSw) = 

W = IVinit 

Vn{Sr) = 1^' 

r = f„(,+i) 

^li+\ 

€ — 'Wji(j) 

[Vi 

W = Wnii) 

[yN 

^ “ ^assert 

V2N+2 

^ ~ ^assert 






We construct a complete set of T'-literals (either { e H„ or e H„ for all £ e This 
will correspond to Mn before abstraction. For any literal £ over TV or T$ atoms, we evaluate £ 
w.r.t. y and v„ to assign it in H^, i.e. £ e H„ if y, |= £. For atoms over TV, we include the 
literals in H„ (defined in section lU. For all other TV equalities £ in cp^, we include -<£ e H„. We 
now let Mn = Ff„®. 


Proof of Lemma \4J\ Since -^L„ c Hn and Mn = Hji®, -iL® c Mn- We now show that for each 
£ e -iLn, we can extend Vn to a new assignment so that p, |= h for all h e Hn \ {£]■ For 
brevity, we denote by 4 = (rv,„,j, = o), f = +1 = for z e 1... N - 1, 4sserfi = 

+ 1 - and £assert2 = > N). 


v^»(rw) = 


1 

7 + 1 
N + 1 


r = f„(i) 
r = rn(j) 

^ ~ ^assert 


(o 


yf^sertl _ 


7 

N-t-l 


r = r„(i) 

f = rny) 

^ ~ ^assert 


V^(rVr) = • 


(rv,) = 


r = r„(i) 

'' = < i 


k+1 r = rn(k),k>i 

TV -f- 1 Y = Yassert 


r = r„(i) 

J' = rn(j) 


77 f" — ^assert 


We omit from the 7V-constants 0,..., N -f 1 above. It is now that case that q, \= h for all 
h e Hn\{£]- Thus Hn \ {£] is satisfiable modulo T. As every subset of Hn excluding exactly one 
literal in -iL„ is satisfiable modulo 7~, ^Ln is the unique minimal 7~-conflict in Hn- Thus TVIn is 
a critical assignment. □ 

Proof of Theoreni \4.3\ We extend Vn to assign sup^ to match ct(7t): V 7 i(sup^^jj|) = V 7 i(Ca,j„„), 
v„(sup,^^.j) = Vji(Ca,„,j_i)), and v„(sup,__^^^) = Vji(Ca,„,j^). We follow the same construction of 
Hn, Mn, v^, and Q as before for (p^- Q is a set of non-interfering critical assignments for (p^- □ 
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B Time and Memory Usage 

Elapsed time and memory usage for fkp2IS13-unsat benchmark; TIMEOUT = 1 hour. 
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0.0 

T 

0.00 
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